Video Coding Basics

Video coding is a very wide and complicated field with a lot of effort spent throughout the last decades.This article will not try to cover everything but only the things necessary for flawless operation of Video Sync. It also doesn't try to be absolutely mathematically or technically correct. Furthermore, it will try to impart a certain amount of basics.

When working with a software like Video Sync, a certain knowledge about video codecs and related topics is preferable. You don't need to know mathematical or technical details, but knowing a couple of buzzwords helps a lot. This article will explain some of the most common terms and clarify some common misunderstandings.

Video Codecs

Encoding movie frames with the encoder as part of the codec {width=60%}

A video codec describes how video data is processed when writing to (encoding) or reading from (decoding) a movie file. The word 'codec' originates from '(en)coder' and 'decoder'.

Almost all common codecs are lossy - some more than others - to reduce the amount of data but this does not necessarily mean a visible loss in quality.

In video coding, there are two groups of codecs. Intraframe-only and interframe codecs.

Intraframe-only Codecs

The first kind is pretty straight forward: every frame of the movie is stored directly in the movie's data block and decoded one after the other during playback. Codecs of this type typically produce large movie files as every movie frame must be stored in its entirety.

Intraframe-only codec with only full frames {width=80%}

The most common intraframe-only codecs used nowadays are: DV and Motion JPEG for SD and Apple Pro Res, Avid® DNxHD and AVC Intra for HD material.

Interframe Codecs

Interframe codecs in contrast work with so-called "Groups of Pictures" (GoP). Generally speaking, this means that not every frame of the movie is encoded directly but rather more just every 5th, 10th, 20th ... frame (depending on the encoder setting). The fully encoded frames are called I(ntra)-frames or keyframes.

Interframe codecs with different image types {width=80%}

In between two I-frames, just the "difference" to the preceding frame is computed and stored. With this procedure it is possible to omit a lot of data. This drastically decreases the resulting file size of movies encoded with such codecs. That's also the reason why movies encoded with interframe codecs are so popular. H.264 is today's most commonly used interframe codec.

Pros and Cons

While intraframe-only codecs like Apple Pro Res for example produce way larger files, they don't require a lot of CPU when being played back as the frame data can simply be read and decoded without a lot of processing that needs to take place.

Interframe codecs, such as H.264, produce much smaller files but require a comparably high amount of CPU power as more data needs to be processed. You have to choose wisely what's more important for your particular use case. (Hint: choose i-frame only codecs!)

Video Sync and ADR Master can handle interframe files properly but the performance will be better with intraframe only files.

Ask your editor for a file encoded with an I-frame only codec like Apple Pro Res. The files will be way larger regarding file size but they provide the best playback and sync quality with Video Sync.

Container Formats

Simply encoding the video data leaves you with a block of raw data. To be able to use it, it must be put into a file along with some other information so the player can read all the data back properly. That's what a container is used for. A container specifies the structure of the information stored inside a file. It also defines which types of media and/or metadata can be embedded.

Imagine a simple container that can hold movie and audio data. When filled, it might look like this:

Example of the internal structure of a movie container {width=50%}

Ok, so we have a video and audio track. This seems sufficient at first, but what if we want to embed a second audio track or some more metadata like subtitles or timecode? Seems as if the container can't handle that. Not all containers can handle everything. This link provides more information about which container formats are capable of holding which audio and video codecs as well as other metadata.

What you can see is that the QuickTime container format has "Yes" in almost all columns of this table making it a very versatile container supporting suitable for post production work. This is probably the reason why this container is still so popular in the movie industry (despite its age - more than 20 years).

While Apple's QuickTime playback engine is deprecated since OS X 10.8, this does not affect the container format. The MOV container still is fully supported by Apple.